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MANPOWER.  PERSONNEL,  TRAINING,  AND  SAFETY  GUIDANCE  AND 
CONTROL  FOR  WEAPON  SYSTEM  ACQUISITION^ 


1.  INTRODUCTION 

It  is  difficult  to  get  manpower,  personnel,  training,  and  safety  (MPTSS)  issues  considered  at 
an  early  stage  during  weapon  system  design.  Government  acquisition  teams  sometimes  provide  very 
little  guidance  about  MPT4S  issues  because  of  uncertainty  about  the  Lind  of  guidance  they  shoula 
provide  and  reluctance  to  interfere  with  contractor  operations.  Contractors  are  almost  forced  to 
dictate  MPT4S  requirements  under  such  circumstances.  One  of  the  reasons  that  this  situation 
occurs  is  that  the  Government  acquisition  team  does  not  have  enough  information  to  provide  all 
the  guidance  that  is  needed. 

On  the  assuiH)tion  that  experience  is  the  best  basis  for  facilitating  MPTSS  decisions, 
experience-based  recommendations  were  collected  from  the  literature  as  well  as  from  experts  in 
the  field,  and  documented  as  recommendations  in  the  paper  that  follows.  Information  alone, 
however,  will  not  solve  the  problem.  Organizational  systems  changes  (improved  analysis 
capabilities,  improved  communication  and  incentive  systems,  and  new  organizational  structures) 
are  also  needed.  The  paper  is  thus  intended  for  consideration  by  policy  and  decision  makers  as 
well  as  by  Government  and  contractor  personnel  who  work  on  the  development  of  new  weapon  systems. 


II.  THE  ISSUES  INVOLVED  IN  GOVERNMENT  GUIDANCE 
AND  CONSTRAINTS 

In  weapon  system  design,  the  most  important  priority  Is  that  the  system  perform  as  required. 
Other  considerations,  such  as  MPTSS  requirements,  are  of  secondary  importance.  There  is  a  lot  of 
merit  in  these  priorities,  since  it  would  be  very  wasteful  to  develop  a  comprehensive  MPTSS  plan 
for  each  strawman  version  of  a  weapon  system  as  it  goes  through  tne  early  concept  exploration 
stages.  One  could  conceivably  develo,.  30  MPTSS  plans,  none  of  which  would  ever  te  used  because 
the  30  strawman  weapon  systems  for  which  the  MPTSS  plans  were  designed  will  never,  in  fact,  be 
developed.  It  is  only  the  approved  weapon  system  design  that  will  actually  need  MPTSS  plans,  and 
these  plans  will  probably  go  through  several  iterations  before  they  settle  down. 

There  is,  however,  another  side  to  this  story.  Assuming  that  the  MPTSS  plans  are  not  taken 
seriously  until  the  31st  iteration,  problems  are  likely  to  occur.  In  the  first  place,  the 

hardware  system  that  "works"  me^y  not,  in  fact,  perform  as  required  if  MPTSS  factors  are 
considered  to  be  of  secondary  inportance  during  the  early  stages.  Assuming  that  the  system  does 
indeed  meet  expectations,  the  Government  may  find  itself  forced  to  accept  a  plan  that  is  not 

realistic  in  terms  of  the  available  resources,  or  the  Government  could  fine  that  there  is  not 

enough  time  or  money  to  develop  the  MPTSS  systems  (e.g.,  expensive  simulators)  that  are  needed 

before  the  system  is  scheduled  to  become  operational.  So,  tfie  Government,  rightly  or  wrongly, 
encourages  contractors  to  develop  MPTSS  plans  early  in  the  we-spon  system  acquisition  process 
(HSAP). 


®The  opinions  expressed  in  the  paper  are  those  of  the  authors  ana  do  not  necessarily 
reflect  an  official  position  of  the  Department  of  Defense  or  the  United  States  Air  Force. 
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The  amount  of  control  that  should  be  exercised  is  controversial.  Too  uuch  Government  control 
becomes  excessive  interference  that  can  stifle  the  contractor's  creativity  or  force  the 
contractor  to  design  a  system  in  one  particular  way.  It  is  always  possible  that  the  contractor 
might  have  used  a  different  approach  that  could  have  saved  the  Government  millions  of  dollars  or 
been  several  times  more  effective  if  fewer  restrictions  had  been  imposed.  At  the  other  extreme, 
lack  of  Government  constraints  can  become  equally  deplorable,  since  the  contractor  could  waste 
millions  of  dollars  designing  something  that  is  prohibitively  expensive  or  cannot  be  used  because 
the  needed  MPT&S  systems  are  not  available. 


III.  RECENT  DEVELOPMENTS  IN  CONTROLS  OVER  HPT&S  DECISIONS 

Need  for  Early  MPT&S  Decisions.  Several  years  ago,  a  number  of  advisory  groups,  including 
the  General  Accounting  Office  [1]  and  the  Defense  Science  Board  [2j  urged  the  Government  to 
consider  MPTgiS  factors  at  an  earlier  point  in  the  WSAP.  In  response,  the  military  services  made 
a  number  of  efforts  to  change  their  procedures,  but  the  initial  results  were  not  always  fully 
satisfactory  [3,  4].  Many  problems  can  occur  when  MPT&S  decisions  are  not  made  early  in  the 
WSAP,  and  the  challenge  of  MPT&S  integration  was  addressed  in  many  different  ways  [5j. 

A  good  example  of  the  need  for  early  decisions  is  in  the  area  of  job  aids.  The  ready 
availability  of  microcomputers  makes  it  possible  to  modify  MPT&S  requirements  extensively  by 
using  job  aids  and  expert  systems.  Job  aids  can  decrease  the  number  ot  maintainers  who  are 
required,  change  high  skill  level  jobs  to  low  skill  level  jobs,  decrease  or  change  the  training 
requirements,  and  convert  unsafe  conditions  into  safe  ones.  As  pointed  out  by  Lineberry  [6j, 
"...guidance  with  job  aids  should  always  be  the  choice,  unless  key  factors  contra-indicate, 
because  job  aids  generally  cost  less  to  develop  than  instruction,  are  easier  to  revise  when 
performance  requirements  change,  reduce  the  time  to  achieve  on-the-job  performance,  and  are  not 
subject  to  foi'getting"  (p.  15).  Booher  [7]  has  provided  a  nine-step  selection  algorithm  for 
identifying  the  most  appropriate  job-performance-aid/training  combination.  These  decisions  must 
be  made  early,  since  the  job  aids  and  expert  systems  could  be  built-in  and  become  part  of  the 
equipment. 

Control  Through  Procedural  Guidelines.  All  three  services  have  developed  procedural 
guidelines  for  controlling  MPT&S  decisions  during  the  various  stages  of  the  WSAP.  The  Navy 
developed  a  system  called  HARDMAN  [8,  9]  (for  Hardware  and  Manpower  Integration),  which  was 
originally  cased  upon  'ome  early  Air  Force  work  in  this  area  [10,  11,  12,  13J.  The  Army  has 
adopted  similar  techniques  based  upon  an  early  version  of  the  Navy  system  [14],  and  has  recently 
expanded  this  approach  to  include  even  more  areas  of  responsibility  as  part  of  a  program  called 
MANPRINT  (for  Manpower  and  Personnel  Integration)  [15].  Recent  evaluations  indicate  that  these 
procedural  guidelines  are  working  reasonably  well  [16,  17,  18],  although  there  were  a  number  of 
initial  problems  in  getting  the  systems  implemented. 

Control  Through  Data  Item  Descriptions  (DIDs).  Another  approach  to  control  is  the  use  of 
standardized  Data  Item  Descriptions  (DIDs)  which  contain  detailed  descriptions  of  the  kind  of 
MPT&S  plans  that  are  to  be  provided  by  the  contractor  [19,  20,  21].  The  DIO  needs  vary  from  one 
stage  of  the  WSAP  to  another.  For  example,  the  Navy  [21]  has  one  MPT  concept  DIO,  a  separate  MPT 
resource  requirements  DID,  and  a  third  MPT  data  report  DID.  Although  revisions  to  these  010s  are 
not  permitted,  portions  of  the  DIDs  can  be  deleted  to  meet  the  needs  of  a  specific  weapon 
system.  The  advance  thinking  in  these  OIOs  about  what  the  Government  should  require  at  va-ious 
WSAP  checkpoints  can  be  very  useful,  even  when  the  original  DID  cannot  be  used. 


IV.  SPECIFIC  WEAPON  AND  AGGREfiATE  SYSTEMS  GUIDANCE 


Guidance  and  control  are  needed  at  the  specific  weapon  system  design  and  aggregate  system 
levels. 

At  the  specific  weapon  system  design  level,  the  major  issues  and  concerns  are  ways  of 
influencing  the  design  of  a  weapon  system  ano  facilitating  cost-effective  performance  of  the 
personnel  assigned  to  it.  Qualitative  and  quantitative  MPT&S  requirements,  key  design 
characteristics  for  manning,  job  aiding,  system  maintenance,  supporting  job  structures,  and 
training  -  all  of  these  must  be  evaluated  with  respect  to  optimum  MPTiS  performance  for  a 
specific  weapon  system.  These  analyses  must  be  closely  coordinated  with  human  factors 
engineering  specialists.  Logistics  support  guidance  is  especially  inportant,  since  it  deals  with 
how,  where,  and  when  the  new  weapon  system  will  be  operated,  maintained,  ano  supported.  Examples 
of  important  logistics  guidance  decisions  art:  disperseo  basing;  maintenance  concepts  and  the 
number  of  different  levels  of  maintenance;  operational  temperatures;  and  the  use  of  dedicated 
crew  chiefs.  Another  important  issue  at  the  weapon  system  design  level  is  the  neeo  to  establish 
an  MPTSS  baseline  for  determining  the  impact  of  proposed  design  changes. 

Aggregate  MPT  AS  systems  combine  information  from  several  different  weapon  systems  and  jobs 
and  examine  MPT4S  policy  issues  from  an  organizational  unit,  major  command,  and/or  military 
department  point  of  view.  In  aggregate  systems,  the  major  issues  are  the  availability  ano 
affordability  cf  MPTSS  options  in  the  context  of  the  total  force  structure  and  all  of  the 
external  demands  that  are  made  upon  it.  The  important  objectives  are  to  avoid  disconnects  and 
unexpected  consequences  for  MPTSS  subsystems  in  future  years  [5j.  Other  issues  at  the  aggregate 
systems  level  are  cross-utilization  of  information,  reduced  overhead  requirements,  ano  policy 
decisions  to  redesign  or  restructure  occupational  specialties.  These  analyses  at  the  higher 
command  level  need  to  be  continuously  transmitted  to  specific  proouct  divisions  for  further 
planning  and  implementation. 


V.  WEAPON  SYSTEM  DESIGN  GUIDANCE 

HPT&S  Guidance  on  the  Way  that  Tasks  are  Assigned.  One  major  MPT&S  impact  of  weapon  system 
design  guidance  is  the  way  in  which  tasks  ano  duties  are  assigned  to  the  total  {operator, 
maintenance,  and  support;  civilian,  military,  and  contractor  support)  man-machine  system  in  order 
to  make  the  weapon  system  operational.  The  constraints  (e.g.,  operator  maintainability,  limits 
on  mean  time  between  failures)  have  important  implications  for  the  assignment  of  tasks  to  humans 
or  machines,  the  cost  effectiveness  of  the  manned  equipment  system,  tne  effectiveness  of  the 
multipurpose  work  group  to  which  the  individuals  belong,  ano  the  extent  to  which  that  particular 
job  assignment  makes  an  individual  more  useful  in  future  assignments. 

One  of  the  key  issues  in  M°TSS  system  design  is  the  amount  and  kind  of  specialization  in 
jobs.  On  those  occasions  when  a  single  weapon  system  will  utilize  all  the  time  of  the 
responsible  personnel,  the  job  design  considerations  are  relatively  straightforward  [2zj.  What 
usually  happens,  however,  is  that  many  personnel  are  involved  in  each  weapon  system  on  a 
part-time  basis.  It  is  possible  to  design  these  part-time  jobs  such  that  personnel  are 
specialized  by  function;  to  establi..h  multifunction  jobs  'r  which  personnel  act  as  generalists; 
and/or  to  use  computer  software  and  job  aids  to  minimize  knowledge  requirements. 

Implications  for  Skill  and  Grade  Progression  Plans.  The  way  in  which  tasks  are  assigned  has 
important  implications  for  skill  and  grade  progression  plans.  Suppose  that  half  the  jobs  in  a 
particular  occupational  specialty  involved  assignments  to  generalist  jobs  and  half  the  jobs 
involved  assignments  to  specialist  jobs.  What  would  this  do  to  career  progression  plans  in  that 
occupational  specialty?  Could  technicians  move  back  and  forth  between  specialist  and  generalist 
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assignments?  Probably  not,  since  the  technicians  wool  a  not  be  qualified  for  many  of  the  tasks 
that  they  would  be  expected  to  perform  in  either  case.  The  situation  is  complicated  by  the  fact 
that  overspecial ized  and  underspecialized  occupational  specialties  already  exist.  According  to 
Eaenfield  [23],  "Today’s  AF  personnel  specialty  classification  system,  as  it  has  evolved  with 
advances  in  weapon  system  technology,  has  resulted  in  over-special ization/job  fragmentation  in 
some  disciplines  and  very  broad-based,  generic  skills  in  other  disciplines.  These  phenomena  have 
resulted  in  a  lack  of  work  force  stability  and  experience,  inefficient  use  of  manpower  resources, 
poor  job  satisTaction  and  declining  retention,  and,  possibly,  an  overstatement  of  manpower 
requirements"  [23,  p.  vii].  These  problems  in  Air  Force  Specialty  Cooes  (AFSCs)  are  a  direct 
result  of  the  way  in  which  tasks  were  assigned  to  personnel  when  weapon  systems  were  designed  in 
previous  years. 

The  Impact  of  System  Utilization  Polic^'  Constraints  on  Job  Design.  Several  kinds  of  system 
utilization  policy  constraints  could  be  imposed  on  the  jobs  that  are  performed  by  operations, 
maintenance,  and  support  personnel.  One  possibility  is  to  require  that  several  functions  be 
performed  by  the  same  person.  For  example,  operators  could  be  required  to  maintain  their 
equipment  to  some  degree  (this  is  quite  common  in  Army  and  Navy).  It  is  also  possible  to  require 
that  the  operators  be  assisted  by  job  lids  and  computers.  Another  possibility  is  to  impose 
limitations  on  the  number  of  personnel  that  can  be  used  when  many  different  functions  must  be 
performed.  This  will  usually  force  the  contractor  and/or  the  involved  Government  agencies  to 
design  generalist  jobs  that  cut  across  traditional  job  specialties.  Another  option  is  to  put 
limits  on  the  amount  of  training  that  can  be  required  or  to  put  limits  on  the  aptitude  or  skill 
levels  of  the  incumbents.  If  the  limits  are  restrictive,  the  contractor  could  be  forced  to 
design  a  system  with  lots  of  job  aids,  computer-assisted  expert  systems,  "black  boxes,'  etc. 
These  tradeoffs  should  be  analyzed  early  in  the  development  cycle  before  resources  are  invested 
in  options  that  will  not  be  utilized. 

VI.  KPTIS  DECISIONS  AT  THE  AGGREGATE  SYSTEM  LEVEL 

Aggregate  Data  Bases  and  Information  Systems.  Each  Service  has  e  variety  of  limited  purpose 
and  aggregate  information  systems  for  MPTSS.  The  major  function  of  these  data  bases  and 
information  systems  is  to  ensure  that  there  are  no  disconnects  or  unexpected  consequences  of 
decisions  at  the  subsystem  level  among  the  organizations  that  are  responsible  for  different  parts 
of  the  MPTiS  System.  For  example,  if  a  new  weapon  system  is  going  to  require  1,000  additional 
fighter  pilots  and  10,000  maintenance  and  support  personnel  during  a  particular  period  of  time, 
it  is  important  that  the  manpower  experts  know  that  the  slots  are  needed  and  distribute  teem  to 
the  right  organizations,  that  the  personnel  experts  set  up  assignment  systenis  that  will  get 
people  to  the  right  places  at  the  right  times,  that  the  training  experts  schedule  the  appropriate 
number  of  trainees  into  the  appropriate  training  pipelines,  and  that  the  safety  experts  certify 
that  the  system  is  safe  and  make  sure  that  the  necessary  safety  regulations  are  issued  ano 
enforced  in  a  timely  fashion..  Aggregate  data  bases  and  information  systems  are  neecea  in  order 
to  do  these  things  [24]  -  ano  they  are  needed  years  in  advance.  Aggregate  oata  bases  are  also 
used  by  top-level  decision  makers  when  choosing  among  competing  systems  for  inclusion  in  the 
future  force  structure. 

The  aggregate  data  bases  could  have  inportant  input-output  relationships  with  job  design  and 
weapon  system  design  decisions.  These  aggregate  data  bases  provide:  informed  inputs  regarding 
the  total  system  consequences  of  specific  weapon  system  designs;  information  about  the  MPTiS 
constraints  that  should  be  iiqposed  upon  weapon  system  design;  ano  long-range  MPTiS  planning 
inputs  to  aggregate  system  plans  for  future  years. 

Manpower,  Aptitude  and  Skill  Level  Constraints.  The  most  likely  constraints  to  be  iiroosed  by 
decision  makers  at  the  aggregate  level  are  constraints  on  the  total  number  of  personnel  at  each 


aptitude  or  skill  level.  As  weapO'>  systems  have  become  more  and  more  complex  and  technical, 
aptitude  I'equirements  -  especially  in  the  electronics  specialties  -  have  increased  from  year  to 
year.  Yet  the  labor  market  is  not  expected  to  change  dramatically  during  the  next  few  year'  and 
wp  will  prooably  have  approximately  the  sime  number  (or  less)  of  high  aptitude  people  in  1995  as 
we  have  today.  When  skills  are  scarce,  who  will  decide  which  weapon  systems  are  really  entitled 
to  higher  aptitude  and/or  skill  level  personnel,  and  which  are  not? 

Each  group  of  weapon  system  designers  tends  to  think  that  their  weapon  system  should  be  given 
priority  over  other  weapon  systems  for  the  small  number  of  military  personnel  who  Qualify  for 
higher  aptitude  jobs.  Yet  we  obviously  cannot  have  job  requirement  profiles  that  do  not 
correspond  with  the  realities  of  the  available  military  oerscnnel  populations  from  which  those 
requirements  must  be  met.  It  '.eems  logical,  ther,  to  impose  constraints  on  the  system 
designers.  For  example,  system  designers  can  be  prevented  from  requiring  that  their  weapon 
systems  be  manned  with  nothing  but  engineering  officers  and  E-7  technic 'ans.  If  the  long-range 
forecasters  are  expecting  to  have  shortages  in  these  categories  -  or,  if  the  jobs  that  would 
prepare  a  person  for  E-7  skills  do  not  exist  (wnich  prevents  personnel  from  gaining  the 
expef^ience  needed  for  higher  level  jobs)  -  the  weapon  system  can  be  desioned  (using  job  aids, 
computer  software,  black  box  replacements,  etc.)  so  that  people  wUh  less  skill,  education,  and 
aptitude  can  do  what  needs  to  be  done.  Moreover,  we  need  to  be  certain  that  these  forecasts  will 
remain  valid  as  systems  go  through  development  and  are  fielded  for  10  years  or  more. 

It  is  clear  that  the  requiremtiits  for  higher  and  higher  aptitudes  cannot  continue 
indefinitely.  The  Army,  which  has  historically  been  most  affected  by  skill  shortages,  is  taking 
an  aggressive  stand  in  tnis  area  with  its  MANPRINT  program  [15].  The  other  Services  will  be 
watching  the  Army's  progress  very  carefully  as  it  develops  new  systems  and  procedures  for 
imposing  manpower  and  skill  level  constraints  on  weapon  system  contractors. 

Training  Budget  Constraints.  It  has  become  commonplace  in  recent  years  to  require  that  the 
contractor  provide  crew  maintenance  and  support  training  for  a  certain  number  of  years  after  the 
new  weapon  system  becomes  operational.  This  has  the  eiiect  of  imposing  faining  budget 
constraints  that  are  likely  to  be  tight  if  the  original  procurement  was  competitive.  3y 
establishing  a  financial  cost  if  tlie  contractor  develops  inadequate  training  systems,  the 
Government  hopes  to  receive  better  quality  training  systems  in  a  more  timely  fashion. 


VI 1.  RECENT  STUDIES  OF  GOVERNMENT  GUIDANCE  AND  CONTROL 

Studies  of  Government  guidance  and  control  have  been  conducted  in  all  three  Services  [25,  <;6, 
27,  28,  29,  30].  Recommendations  regaraing  guioance  and  control  have  also  been  provided  as  a 
result  of  conferences  with  industry  [31].  The  consensus  is  that  the  new  MPTbS  .nanagoment  systems 
(e.g.,  HARDMAN,  MANPRINT)  are  being  used  and  are  having  a  beneficial  effect. 

Most  of  the  problems  that  have  occurred  can  be  attributed  to  less-than-adequate,  biased,  or 
excessive  control  by  the  Government.  The  following  statements  sutmiarize  expert  opinions 
regarding  the  "direct  causes"  of  the  human  factors  and  MPT  problems  that  have  occurred. 


Undercontrol 

There  was  ambiguity  and/or  lack  of  precision  in  describng  required  system  objectives. 

System  description  was  incomplete. 

Task  and  skill  analyses  and  man-machine  tradeoff  studies  were  not  reau'.red  early  enough  to 
affect  basic  systems  parameters. 
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Many  of  ihe  proposed  MPT&S  measures  could  not  be  verified  or  enforced. 

There  was  laxity  in  following  up  and  verifying  human  factors  and  MPT&S  supportability  gcals. 

Test  and  evaluation  plans  did  not  emphasize  maintenance  support  requirements  in  operational 
environments. 

Inadequate  guidance  and  unmeasurable  criteria  were  contained  in  requirements  documents, 

MPT&S  decision  points  and  evaluations  for  new  systems  were  prograinned  without  adequate  test 
or  evaluation. 

Desi'jn  requirements  for  training  equipment  were  very  general  and  incomplete.- 
No  penalties  were  established  for  failure  to  perform  MPT&S  planning. 


Overcontrol 


Some  systems  requirements  were  specified  exactly  when  they  should  have  been  determined  by 
tradeoff  analysis  studies. 

Status  Quo  Approach 

MPT&S  approaches  that  had  worked  for  previous  systems  were  accepted  uncritically  without 
proper  examination  of  the  unique  circumstances  of  the  system  currently  under  development. 

Personnel  characteristics  of  previous  systems  were  assumed  to  be  valid  for  new  systems, 
without  adequate  test  or  evaluation. 

Maintenance  requirements  were  assumed  to  be  met  with  routine  and  standard  maintenance 
procedures  when  other  options  should  have  been  explored. 

Manning  was  by  policy  rather  than  by  requirements. 

Hardware  Bias 


There  was  a  tendency  to  overlook  personnel-oriented  performance  measures  and  man-machine 
tradeoff  studies  in  favor  of  equipment  development. 

In  performance  specifications  there  was  too  much  concentration  on  hardware  ratner  than 
man-machine  performance. 

There  was  a  tendency  to  overlook  human  performance  measures  in  favor  of  hardware-oriented 
performance  measures. 

The  general  attitude  was,  "Let's  worry  about  the  equipment  first;'  we  can  always  get  the 
people  later." 


VIII.  RECOKMENOrO  GUIDANCE  FOR  WEAPON  SYSTEM  DESIGN 

Based  upon  our  analysis  of  the  case  studies  reported  in  the  literature  and  conversations  with 
experts  in  the  field,  a  slightly  different  approach  seems  to  be  needed  at  each  stage  of  the  WSAP 
(see  Table  1). 
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Table  1.  Recommended  Approach  for  Weapon  System  Design 


WSAP  phase 

Present  approach  to 

MPT&S  requirements 

Reconmended  approach  to 

MPT&S  requirements 

Pre-Concept 

Consensus  of  responsible  organ¬ 
izations  that  are  primarily 
responsible  for  the  status  quo 

Creative  analytic  studies  of  MPT&S 
constraint  alternatives,  goals,  and 
issues 

Concept  Evaluation 

Engineering-oriented  trade 
studies 

Total -system-oriented  trade  studies 
(including  MPT&S  alternatives)  for 
both  operators  and  maintainers 

Demonstration-Val  idation 

Budget  is  usually  adequate 
only  for  engineering  system 
improvements 

Adequate  budgets  for  total  system 
inprovements  and  alternate  system 
analysis 

Full-Scale  Development 

Quick  fixes  for  inadequate  or 
underdeveloped  MPT&S  systems 

Evolutionary  changes  only,  since 
MPT&S  system  needs  are  alreatiy 
anticipated 

Production  and  Deployment 

Gradual  evolution  of  MPT&S 
system  inprovements 

Minor  changes  only,  since  MPT&S 
system  needs  are  already  anticipated 

Pre-Concept.  During  the  pre-concept  phase,  the  Government  neeos  some  way  of  specifying 
constraints  without  telling  the  contractor  how  to  design  the  weapon  system.  These  constraints 
are  required  because  of  the  circumstances  under  which  the  weapon  system  would  be  used.  For 
example,  limitations  on  maintenance  manpower  could  be  created  because  of  dispersed  basing 
requirements.  Even  though  these  constraints  are  imposed  by  system  utilization  policies,  it  is 
still  possible  to  give  the  contractor  enough  freedom  to  come  up  with  a  range  of  personnel  mixes 
in  support  of  the  type  of  weapon  system  desireo.  The  contractor  can  be  required  to  conduct 
broad-brushed  total  system  trace  studies  before  recommendations  are  made  regarding  the  design  of 
specific  MPTSS  subsystems. 

Contractors  do  not  want  to  be  perceived  as  "non-responsive. "  They  will  usually  give  the 
Government  what  it  says  is  wanted,  unless  there  are  strong  reasons  to  do  otherwise.  So  the 
Government  acquisition  team  must  be  very  careful  about  what  the  Government  "says"  is  wanted.  On 
the  other  hand,  the  performance  of  work  costs  money  -  and  the  contractors  will  not  perform  work 
that  is  "implied"  or  "seems  to  be"  a  logical  requirement  unless  there  is  an  explicit  requirement 
that  they  do  so.  This  is  especially  true  of  tradeoff  and  sensitivity  studies  for  MPT&S 
alternatives.  It  is  important  that  the  requirement  for  such  studies  be  explicitly  stated  in  the 
Request  for  Proposal  (RFP)  when  it  is  issued.  It  is  also  important  that  the  tradeoff-thinking 
implicit  in  such  a  requirement  not  be  negated  by  other  requirements  in  the  RFP,  The  Government 
should  not  ask  the  contractor  to  plan  and  conduct  manning  tradeoff  studies,  for  example,  while 
simultaneously  requiring  that  the  weapon  system  be  operated  by  a  two-person  crew.  Another 
important  point  to  remember  here  is  that  good  MPTSS  systems  will  not  be  free.  If  the  Government 
wants  high  quality  MPTSS  systems,  it  must  pay  for  quality. 
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Too  often,  the  pre-concept  constraints  are  decioeo  upon  by  contacting  the  headquarters 
organizations  with  responsibility  for  each  relevant  area  of  expertise,  and  arriving  at  a 
consensus.  The  time  available  for  studying  these  issues  at  these  heaoquarters  organizations  is 
rarely  adequate  for  a  comprehensive  study  of  constraint  alternatives.  The  headquarters 
organizations  cannot  alweiys  be  as  future -oriented  as  they  should  be,  since  they  are  very  busy 
trying  to  keep  track  of  the  status  quo;'  nor  do  they  usually  have  available  the  kind  of 
long-range-oriented  analytic  capabilities  that  are  needed;  and  the  aggregate  data  bases  that  are 
needed  to  justify  constraints  are  not  always  available. 

Concept  Evaluation.  The  typical  concept  evaluation  trade  stuoy  at  the  present  time  is 
engineering-oriented.  What  is  needed  instead  are  total-sy stem-oriented  MPT4S  trade  studies 
I  (including  both  operator,  maintainer,  ana  support  personnel)  in  which  man-machine  tradeoffs  are 

I  considered.  These  tradeoff  studies  cannot  be  permitted  to  become  "pencil-whipping"  exercises  in 

!  which  evaluations  are  based  upon  superficial  analyses  of  alternatives  that  are  not  really 

I  competitive.  In-house  Government  expertise  and  independent  quality  control  checks  are  needed  in 

j  order  to  make  certain  that  the  concept  evaluation  trade  studies  are  well  conducted  and  taken 

1  seriously. 

Demon stration-Val idation.  A  coninon  conclusion  after  competitive  procurements  are  awarded  is 
that  the  demonstration- validation  budgets  are  adequate  only  for  engineering  system  improvements; 
MPT&S  plans  (and  possibly  logistics  and  maintenance  plans  as  well)  are  often  curtailed  because 
the  engineering  budgets  were  underestimated,  it  may  be  hard  for  the  Government  to  do  this  at 

times,  but  someone  needs  to  step  in,  evaluate  the  plans,  recognize  that  the  budget  is  inadequate, 

and  take  whatever  steps  are  needed  to  ensure  either  that  tha  budget  is  increased  or  that  the  work 
plans  are  modified  to  redefine  the  system.  This  may  be  difficult  to  do  when  the  company  has  a 

fixed  price  contract  and  there  are  alreac(y  cost  overruns  and  schedule  slippages  -  but  someone 

must  do  it  if  MPT4S  factors  are  to  be  given  the  weight  that  they  deserve. 

i 

1  Full-Scale  Development.  The  typical  approach  during  full-scale  development  is  that  of  quick 

[  fixes  to  resolve  MPT8S  oversights.  Evolutionary  changes  are  to  be  expected;  however,  very  few 

quick  fixes  should  be  needed  if  the  MPTAS  requirements  are  properly  anticipated  (and  given 
I  realistic  budgets)  during  earlier  WSAP  phases.  The  MPTAS  efforts  during  full-scale  development 

should  be  devoted  to  refining  the  MPT  products  developed  earlier  (numbers,  skill  levels,  tasks, 
and  training  analyses).  Test/evaluat  and  validation  of  these  MPTAS  projections  need  to  be 
programmed.  In  addition,  evaluatioi.  of  training  development,  training  media  and  materials 
(formative  and  summative)  will  be  a  major  activity. 

Production  and  Deployment.  When  the  new  weapon  system  is  actually  deployed,  there  may  still 
be  a  need  for  some  quick  fixes,  but  one  thing  is  certain;  If  the  MPTAS  requirements  were  not 
understood  before,  they  are  about  to  become  understood  in  a  hurry.  For  obvious  reasons, 
operational  personnel  are  active  proponents  of  inprovenents  that  would  make  the  system  more 
effective  and  cost-efficient.  MPTAS  systems  will  consequently  improve  during  production  and 
deployment  in  an  evolutionary  way  as  fast  as  circumstances  will  permit.  There  is  nothing  wrong 
with  this  process,  and  nothing  wrong  with  the  importance  attributed  to  MPTAS  factors  (at  long 
last).  Ideally,  however,  the  MPTAS  needs  would  have  been  adequately  anticipated  in  previous 
stages,  and  little  change  should  be  needed  during  the  production  and  deployment  stages. 

IX.  CONSTRAINTS  ON  THE  MPTAS  PROCESS 

Weapon  System  Design  Constraints.  Almost  everyone  is  willing  to  agree  that  MPTAS  utilization 
policy  and  task  assignment  constraints  should  be  developed  and  imposed  during  the  pre-concept  and 
concept  evaluation  phases.  Unfortunately,  the  Government  personnel  responsible  for  developing  a 
weapon  system  usually  do  not  have  a  clear-cut  idea  as  to  what  these  constraints  should  be. 
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System  utilization  policy,  skill  level,  and  task  assignment  constraints  are  haro  to  specify  when 
the  exacc  nature  of  the  equipment  is  unknown  and  the  equipment  developers  and  the  MPT4S  experts 
are  in  separate  organizations.  They  are,  however,  no  more  difficult  to  specify  than  the 
equipment  options  under  consideration.  The  most  important  impact  cf  system  utilization 
constraints  is  on  the  assignment  of  functions  to  man  or  machine.  New  and  iirproveo  total  system 
analysis  techniques  are  needed  to  evaluate  the  pros  and  cons  of  assigning  tasks  to  human 
personnel,  to  machines,  to  human  personnel  equipped  with  jot  aids,  to  specialists,  to 
generalists,  etc.  Logistics  system  constraints  are  especially  important.  txamples  are:' 
dispersed  base  locations;  maintenance  levels;  dedicated  crew  chiefs;  requirements  for  operator 
maintainability;  limitations  on  the  number  of  maintenance  personnel  available  to  support  a 
system;  and  requirements  for  the  consideration  of  machine-assisted  alternatives  that  would  limit 
crew  size.  Clearly,  the  MPT&S  developers  need  to  work  closely  with  human  factors  engineers  in 
order  to  deal  with  these  constraints. 

Aggregate  System  Constraints.  Aggregate  system  constraints  usually  derive  from  the  projected 
availability  of  personnel  at  particular  skill  levels,  the  feasibility  of  establishing  new 
occupational  specialties  to  support  a  particular  weapon  system,  acceptable  training  times,  etc. 
It  is  important  that  this  guidance  be  provided  in  a  flexible  format  that  permits  tradeoff 
studies.  It  is  also  possible  to  be  more  directive.  One  Arn\y  general  recently  directed,  for 
example,  that  the  Army  establish  a  Design  for  Discard  (DFD)  program.  The  emphasis  in  DFD  was  to 
be  "innovative  design  to  reduce  the  cost  of  discard"  rather  than  repair  cost  analysis  or 
classical  engineering  approaches  [32].  The  general  decided  on  this  approach  because  of  manpower 
projections  that  fewer  people  with  higher  skills  would  be  available  when  needed  and  excessive 
"tooth  to  tail"  (i.e.,  combat  to  logistics  support)  ratios.  Ideally,  however,  aggregate  data 
bases  would  be  used  to  provide  guidance  without  ruling  out  viable  alternatives  when  new  weapon 
sysums  are  designed. 


X.  THE  NEED  FOR  ENHANCED  ANALYTIC  CAPABILITIES  WITHIN  GOVERNMENT 

It  is  easy  to  tell  Government  representatives  that  they  should  provide  more  information  about 
system  constraints,  it  is  not  easy  to  tell  them  how  to  do  it.  Nor  is  it  really  clear  who  shoulo 
conduct  the  quality  control  checks  ana  provide  the  weapon  system  designers  with  the  kind  of 
guidance  that  is  needed. 

Many  analytic  procedures  already  exist  to  justify  constraints  at  the  weapon  system  design 
level.  This  is  not  as  true  of  constraints  that  logically  originate  at  the  aggregate  system 
level.  Neither  the  Government  contract  monitors  nor  the  weapon  system  contractors  are  likely  to 
have  the  expertise  that  is  needed  to  say  what  these  constraints  should  be.-  They  rarely  have 
access  to  long-range  forecasts  and  long-range  plans;  they  rarely  have  the  "big  picture";  and  they 
are  not  supposed  to  establish  policy. 

Each  military  department  has  "stuaies  and  analysis"  groups  that  conduct  constraint-orienteo 
studies  of  the  type  that  is  needed  -  but  they  are  rarely  available  to  study  specific  weapon 
system  constraints  on  short  notice.  New  data  bases,  analytic  methods,  and  study  groups  seem  to 
be  needed  ’n  order  to  help  expedite  this  process.  Important  tools  and  guidelines  needed  by  MPT^IS 
study  groups  are:  ways  of  stating  MPTiS  requirements  in  terms  of  criteria  that  can  easily  be 
measured;  ways  of  dealing  with  the  interfaces  between  subsystem  data  bases;  and  ways  of 
forecasting  the  impact  of  weapon  system  design  decisions  on  MPTSS  criteria  at  early  stages  during 
the  design  process.  The  data  bases  ana  methods  should  be  a  computerized  system  that  would 
include  systems  characteristics,  logistics,  MPTAS  factors,  warfighting  capability,  and  costs.; 
The  new  data  bases  and  analytic  methods  should  assist  and  interact  with  the  MPT&S  analyst  in  a 
"decision  support"  mode  [33],  and  help  get  his  or  her  inputs  considered  during  relevant  facets  of 
the  weapon  system  design  process,  hopefully  including  an  interface  with  the  computer-assisted 


design  (CAD)  p.ocess.  The  system  should  be  capable  of  simulating  wartime  scenarios  given  various 
inputs  (reliability  rates,  numbers  of  people,  etc.).  The  system  should  also  permit  various 
levels  of  analysis  -  top  level  as  well  as  more  specific  options. 

The  new  guidelines  and  decision  aids  are  needed  to  make  it  easier  to  model  a  new  system  in 
terms  of  its  complexity,  types  of  components,  and  MPT&S  requirements.  Analytic  methods  that  are 
capable  of  evaluating  tradeoff  decision  options  and  identifying  the  best  options  for  further 
exploration  are  also  needed.  Given  these  decision  aids  ard  data  boses,  early  budgeting  and  MPT&S 
requirements  could  be  based  on  historical  records  end  r.-owth/cost  curves.  Th''se  early  MPT&S 
estimates  could  then  be  refined  (possibly  using  computer-assisted  update  systems)  as  more 
specifics  are  learned  during  the  design  and  developmental  processes. 

Unified  data  bases  [34]  seem  to  be  logical  prerequisites  for  these  MPT&S  decision  support 
systems  -  but  a  lot  of  work  still  needs  to  be  done,  in  spite  of  the  many  procedural  guidelines 
that  already  exist.  We  ^re  a  long  way  from  the  system  described  in  the  preceding  paragraphs. 

XI.  THE  HEED  FOR  INCENTIVES 

To  make  the  MPTiS  system  work,  it  is  possible  to  use  the  same  incentives  approach  that  was 
used  with  the  Air  Force  Reliability  and  Maintainabi licy  (R8M  2000)  program  that  was  signed  into 
action  on  1  February  1985  [35,  36].  This  would  require:  clear  statements  of  MPT&S  needs  in 
official  requirements  documents  throughout  the  entire  WSAP;  quantitatively  stated  requirements  to 
select  MPT&S  systems  that  are  systems-effective  and  cost-efficient;  improved  source  selection 
procedures  that  would  give  more  weight  to  the  past  MPT45  record  of  the  companies  that  are  being 
evaluated;  the  documentation  of  “lessons  learned"  regarding  MPT&S  system  tradeoffs  and  their 
dissemination  to  all  involved  contractor  organizations  and  Government  agencies;  contract 
incentives  and  warrantifs  that  would  guarantee  satisfactory  MPTI>S  systems  for  a  given  number  of 
years  after  the  sysfan  becomes  operational;  contract  evaluation  points  that  are  timed  to 
correspond  with  the  satisfactory  development  of  MPT&S  systems;  specific  requirements  for 
timeliness  and  ready  accessibility  of  needed  MPT&S  products;'  specific  requirements  for  field 
evaluations  of  MPTtS  systems  before  the  implementation  phase  is  reached;  and  a  DOD-wide 
coordinating  group  that  would  ensure  that  new  ideas  for  improved  MPT&S  systems  are  put  to  work  in 
an  expeditious  fashion. 

A  similar  set  of  incentives  is  needed  to  avoid  disconnects  and  unexpected  consequences  within 
Government  organizations.  For  the  contractors,  money  is  the  best  incentive.  For  Government 
MPT&S  managers,  the  best  incentive  is  to  provide  prompt  cost-effectiveness  feedback  to  the 
managers  of  those  who  make  the  planning  decisions.  Qualified  evaluators  and  enhanced  study 
analysis  capabilities  are  needed  to  provide  the  kind  of  feedback  that  is  needed.  General  officer 
support  is  needed  to  make  certain  that  the  evaluations  are  taken  seriously. 

XII.  THE  NEED  FOR  CENTRALIZED  HEADQUARTERS  COORDINATION  GROUPS 

Although  all  three  Services  have  established  headquarters  focal  points  for  M^T&S  systems,  the 
authority  and  the  resources  allocated  to  these  headquarters  groups  have  not  always  been 
adequate.  The  current  headquarters  staff  groups  in  the  Air  force  do  not  have  enough  influence  or 
resources  to  insist  upon  or  support  analytic  studies  of  system  utilization  policies  and  aggregate 
system  constraints,  for  example. 

Since  all  three  Services  are  working  this  problem  area  using  similar  policies  and  procedures, 
it  may  be  desirable  to  set  up  a  DOD-wide  Headquarters  Coordination  Group  for  MPT&S  systems.  An 
organization  along  these  lines  ilready  exists  in  the  training  area  -  the  Training  and  Performance 


Data  Center  (TPOC)  [37].  It  is  possible  that  TPDC  could  be  modified  to  give  it  a  broader 
perspective  so  that  it  could  accept  more  responsibilities  in  the  MPT&S  area. 

Even  if  the  TPDC  role  is  broadened,  however,  a  strong  headquarters  focal  point  for  MPT&S 
factors  is  needed  within  each  military  department.  It  is  very  important  that  headquarters 
coordinators  have  the  authority  to  direct  that  MPT&S  policies  and  aggregate  systems  guidance  be 
followed  by  lower  echelons.  The  need  for  such  a  group  in  the  Air  Force  was  recognized  in  the 
recent  Akman  Associates  report  [21]  on  the  design  of  Air  Force  systems  for  Readiness  Achieved 
through  Manpower  Personnel,  Requisite  Training,  and  Safety  (RAMPARTS).  An  important  proposal  in 
their  report  was  that  a  strong,  centralized  office  be  established  within  the  Air  Force. 

One  of  the  most  important  objectives  for  new  organizational  structures  is  improved 
communications  between  weapon  systems  designers,  data  base  designers,  and  experts  at  the 
aggregate  systems  level.  People  need  to  talk  to  people  -  to  ask  questions,  get  expert  advice, 
and  let  the  experts  know  how  well  their  recommendations  worked  out;'  and  these  communications  need 
to  take  place  quickly.  Designing  communication  systems  of  this  type  is  an  important  challenge 
for  those  who  would  establish  new  organizations  to  facilitate  the  MPT&S  planning  process. 

XIII.  -DO'S  AND  DON'TS” 

We  prepared  two  lists  of  "Do‘s“  and  "Don'ts":  one  for  Government  acquisition  teams  (Table  2) 
and  one  for  teams  of  contractor  personnel  (Table  3).  We  then  sen;  preliminary  drafts  of  Tables  2 
and  3  for  review  by  approximately  20  experts  in  the  field.  As  a  result  of  their  comments,  some 
additional  items  were  added  to  the  lists,  and  some  of  the  original  items  were  revised  or 
deleted.  The  editorial  decisions  are  ours,  however;  so  the  two  lists  do  not  represent  a 
consensus. 

XIV.  CONCLUSIONS 


Existing  Government  guidelines  and  constraints  for  those  responsible  for  MPT&S  factors  in 
Government  acquisition  teams  are  not  working  well.  There  are  many  instances  of;  undercontrol; 
overcontrol;  too  much  use  of  a  status  quo  approach;  and  a  strong  hardware  bias.  Providing 
experience-based  guidance  to  Government  and  industrial  personnel  will  go  a  long  way  towards 
improving  the  situation,  but  it  is  not  enough. 

Satisfactory  guidance  and  control  are  not  likely  to  be  forthcoming  unless  the  following  steps 
are  taken:  the  development  of  enhanced  analytic  capabilities  that  can  analyze  system  utilization 
policies  and  make  tradeoffs  between  man  and  machine  in  performance  of  system  tasks;  the 
establishment  of  interactive  communication  channels  between  experts  in  weapon  system  design  and 
aggregate  system  constraints;  the  establishment  of  incentive  systems  that  will  reward  both 
Government  and  contractor  personnel  for  giving  greater  priority  to  MPT&S  factors  in  weapon  system 
design;  and  the  establishment  of  a  strong,  directive  headquarters  group  that  can  act  as  an 
advocate  of  total-system-oriented  MPT&S  plans  within  each  military  department. 
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have  MPT&S  supportable  systems  and  those  unnoticed, 

that  may  have  problems. 
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Re<}uirc-  cost-effectiveness  and  cost-benefit  Don't  accept  MPTSS  justifications  of  Important 

evaluations  of  the  most  important  MPTSS  systems  that  do  not  contain  adequate  cost- 

alternatives.  effectiveness  and  cost-benefit  evaluations. 
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Consider  beginning  training  development  con-  Don't  assume  that  training  development  can  play 

tracting  early  enough  to  allow  development  "catch  up"  to  system  production  using  compressed 


ijrwar-Maru 


(/) 

4^ 

s 

«*» 

Of 

4-f 

- 

k. 

i/t 

u 

r— 

3 

C 

o 

O 

<0 

9 

O 

<Q 

U 

k. 

k. 

JC 

4-» 

4^ 

4-> 

‘'f  C 

>1 

o 

Of 

c 

<Q 

Of  O 

c 

> 

o 

Of 

2 

JZ 

4^  ^ 

c 

•r- 

u 

4^ 

9  4J 

o 

** 

3 

t/> 

E  10 

V- 

E 

*o 

C 

</> 

•r-  U 

o> 

.a 

Of 

fo 

Of 

4J  -P. 

c 

k. 

x: 

r— 

> 

«/l  4. 

10 

.C 

Of 

u 

a 

•fM 

Of  .r> 

u% 

P 

4^ 

i/% 

4-> 

k. 

3 

Of 

k. 

c 

<T3 

i/%  Of 

Ck  « 

U  (u  ^ 
O  >  <0 
4- 

(/)  4- 
0>  C 

c  Q} 
a  • 
X  K  01 
O  0»  r- 

I— 

<0  lO 


S  O  O 
o  ^  > 

4-»  •p- 

Q> 

I-  k.  -e- 


3 

3 

p 

44 

44 

3 

iA 

w 

ti) 

o 

P 

o 

P 

IO 

o 

44 

k. 

(A 

p 

JZ 

JZ 

IA 

Of 

4-» 

"O 

> 

44 

iff 

44 

44 

P 

a 

C 

c 

O 

*«• 

44 

2 

P 

£ 

44 

o 

U 

P 

— 

44 

a 

u 

E 

2 

c 

Of 

>» 

•p“ 

p 

•o 

3 

P 

o 

m 

c 

o 

a> 

c 

iA 

E 

•P" 

g>— 

u 

c 

IQ 

iA 

3 

10 

44 

iO 

IO 

10 

P 

£ 

3 

3 

k. 

k. 

P 

44 

u 

</) 

O 

44 

•#“ 

f— 

44 

a 

44 

P 

P 

3 

a 

3 

p 

e- 

a 

C 

O' 

C 

o 

"c 

U 

4^ 

3 

o 

P 

•r- 

o 

o 

o 

«/) 

(/I 

PH 

iA 

o 

w 

> 

o 

*0 

o 

a  r->  t/> 

o  a  E 
0) 

k.  0)  ^ 
3  £  O 
O  4^  k. 

o 

4^  a> 

«  o 
JC  u 
^  s 
a>  E 

3  C 

tA  (Q  IQ 
4)  JC 
<0  4^ 

£  3 

4-»  01  O 

•  *»  SZ 

C  (/)  4^ 
O  ‘r- 

0^2 


a 

* 

C 

s: 

>»  in 

o 

to 

U  44 

•r» 

40 

3 

(O  C 

44 

f- 

P 

3  p 

P 

a 

iA 

cr  6 

3 

se: 

o 

p  p 

r— • 

a 

'O  k. 

Id 

o 

10  •r~ 

> 

o 

k. 

3 

p 

a 

P  O* 

v> 

4=  p 

3 

44 

•S  e 

U) 

p 

o  p 

E 

P  44 

P 

3 

a  to 
>» 

44 

to 

p 

5 

p  5? 

to 

>> 

o 

o 

•0 

p 

1— 

if) 

C 

a 

c 

3 

p 

o 

>> 

£ 

o 

3 

u 

44 

a 

P 

k. 

44 

44 

4-> 

p 

JC 

3 

U 

P  k. 

o 

p 

u 

o 

3 

4=  O 

«/> 

(/> 

3 

**  a 

44 

p 

P 

k.  k. 

4-> 

(/>  ^  f 

OO  (/>  4- 


o  • 

s  g 

2  S 

(U  V) 
^  >» 
Q>  lA 
O  U 
U  (/> 

4^  lO  qO 
U  ^ 
10  C  Q. 

^  *0  t: 


P 

44 

Si 

£ 

o 

10 

44 

44 

3 

iA 

44 

iA 

iO 

IO 

P 

> 

3 

£ 

> 

p 

C 

44 

IO 

44 

3 

P 

p 

C 

iA 

k- 

C 

10 

k. 

k. 

P 

3 

P 

44 

a 

3 

44 

iA 

E 

P 

P 

3 

a 

IO 

c 

3 

tf> 

c 

h— 

•r* 

o 

• 

c 

o 

3 

• 

a 

k. 

k. 

p 

3> 

p 

p 

Of 

iA 

£ 

44 

JC 

c 

P 

X 

3 

p 

.o 

P 

• 

44 

Cf> 

k. 

p 

p 

C 

k. 

•r« 

c 

44 

to 

tn 

•e“ 

k. 

(/> 

c 

> 

IO 

3 

V) 

iA 

3 

Q 

p 

p 

3 

44 

C 

O 

P 

44 

3 

k. 

3" 

(/> 

10 

a 

O) 

C 

3> 

P 

o 

c 

*»“ 

c 

P 

C 

3 

3 

u 

c 

p 

3 

IO 

o 

•r» 

C 

to 

o 

£ 

i/%  *0 

4^  f-  k-  • 

<0  r~  Of  01 

iS  fO  > 

O  <t>  «0 
k.  s  k. 
Of  Of 

k-  o>  o>  a 

3  c  c  o 

4>  ♦r- 

C  V)  o 

Ui  3  3  4> 


C 

a  to 

3 

(*- 

p 

O 

44 

o 

•r» 

p 

£ 

3 

44 

C 

3  3 

44 

3 

P 

p 

IO 

C 

44  p 

P 

Si 

k. 

44 

k. 

E 

P 

p 

k. 

k.  k. 

k. 

10 

3> 

p 

k. 

3 

9 

iO  P 

•p* 

3 

C 

3 

3 

44 

a  «4> 

3 

O 

*o 

3 

P 

c 

3 

k. 

o 

£ 

k. 

>»  O 

P 

3 

P 

C 

3 

3 

^  3 

a 

10 

3 

CO 

a 

U> 

C  4^ 

O  C 

f-  Of 

4-»  E 

O  -o  >> 
3  C  O 

O  »0  r— 

o  a 

s-  Of 

a  o 


17 


Document  MPT4S  lessons  learned  so  others  can  Don't  assume  that  MPT4S  problem  histories  won't  be 

benefit  from  this  experience.  repeated. 


Table  3.  "Do's"  and  "Don’ts*  for  Teams  of  Contractors 
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Conduct  human-machine  traaeotf  stuuies  early  Don't  defer  human-machine  tradeoff  studies  until 

enough  to  influence  the  design  of  the  equipment  characteristics  are  relatively  well 

equipment.  established. 
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